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Abstract 

This study investigated whether (a) cooperative learning groups with the highest 
mean levels of knowledge of the research process, as measured via mean midterm and final 
examination scores, produce the best cooperative learning projects, as measured by the 
quality of research article critiques and proposals; (b) degree of heterogeneity is related to 
the quality of these group projects; and (c) size of the group is related to quality of output 
produced. 

Participants were 275 graduate students enrolled in several sections of an 
introductory-level educational research course who, through a modified stratified random 
assignment procedure, formed 70 groups ranging in size from 2 to 7. Using group as the unit 
of analysis revealed a small-to-moderate positive relationship between the mean midterm 
and final examination scores and scores on the article critiques and proposals--the former 
suggesting a “Matthew Effect,” with respect to group outcomes. A positive relationship was 
found between degree of group heterogeneity at the midterm level and scores on the 
projects. Also, a quadratic trend defined the relationship found between group size and 
performance on the article critique. Finally, a treatment (i.e., group heterogeneity level) x 
aptitude (i.e., mean midterm group performance) was found with respect to the article 
critiques produced. 
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Treatment By Aptitude Interactions as a Mediator of Group Performance 
in Research Methodology Courses 

Cooperative learning is one of the most common techniques utilized by educators 
throughout the United States at every tier of the educational process. This method 
represents the instructional use of small groups in which students collaborate either formally 
or informally to maximize their own learning, as well as those of their fellow group members 
(Johnson, Johnson, & Smith, 1991a). Theoretical support for this instructional strategy is 
based on the social interdependence, cognitive-developmental, and behavioral learning 
theories (Johnson, Johnson, & Smith, 1998). 

Social interdependence theory, based on the work of Morton Deutsch and Kurt Lewin 
(Deutsch, 1949; Lewin, 1935), posits that cooperation is the result of positive 
interdependence among individuals’ goals. According to this theory, positive 
interdependence (i.e., cooperation) leads to promotive interaction as students within a 
cooperative learning group encourage and facilitate each group member’s learning and 
output. On the other hand, negative interdependence often results in dysfunctional 
interaction as group members impede and discourage each other’s efforts to perform 
(Johnson et al., 1998). In other words, social interdependence positively influences 
individual interaction with a given situation, which subsequently affects the outcomes of that 
interaction (Johnson & Johnson, 1989). 

Theoretical Perspectives 

According to cognitive-developmental theory, cooperation is paramount for cognitive 
growth. Jean Piaget theorized that when individuals interact with society, positive socio- 
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cognitive contradictions occur that induce a state of cognitive disequilibrium (Johnson et al., 
1998). This disequilibrium, in turn, promotes perspective-taking ability and, hence, cognitive 
development. Dialectical theories, also cognitive-developmental in nature, assert that 
knowledge is social, constructed by society and conveyed to the individual. Experiences 
during adulthood lead to questions, doubts, and contradictions that may culminate in further 
re-organization of thought in which conflicting viewpoints are integrated into a larger 
framework (Kramer, 1983; Labouvie-Vief, 1985). 

Dialectical theories first became popularized when psychologists from the then Soviet 
Union were searching for a model that would be compatible with the Marxist ideology. Its 
major advocate, Lev S. Vygotsky, shortly after the Russian Revolution, proposed a way of 
conceptualizing human development in which mental activities take shape within a social 
framework (cf. Vygotsky, 1978). Vygotsky contended that cooperative efforts to learn, to 
understand, and to solve a wide range of problems are central for constructing knowledge 
and transforming the different perspectives into efficient mental functioning (Johnson et al., 
1998). Vygotsky, like Piaget, believed that learning cooperatively with more-able peers and 
instructors culminates in cognitive growth and intellectual development (Johnson et al., 
1998). Indeed, the cooperative learning concept of scaffolding (i.e., less skillful students 
actively collaborating with more competent peers, thereby enabling the former to develop 
more complex levels of understanding and skill) is a by-product of these cognitive- 
developmental theories. 

Behavioral learning theorists posit that students will maximize their performance 
levels on tasks for which a reward of some sort follows; conversely, students will minimize 
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Treatment By Aptitude Interactions 5 
their efforts on tasks that yield minimal or no reward, or even punishment (Johnson et al., 
1998). Cooperative learning involves the provision of incentives for members of a group to 
collaborate with their group colleagues. Albert Bandura, one of the earliest proponents of 
social-learning theory, emphasized the importance of imitation and modeling in the learning 
process (Bandura, 1969). B.F. Skinner, known for his theory of operant conditioning, 
advanced the use of group contingencies in promoting learning (Skinner, 1938). 

As summarized by Johnson et al. (1998), social interdependence theorists believe 
that cooperation is based on intrinsic motivation induced by interpersonal components, with 
a collaborative desire to achieve being central toward achieving cooperative goals. 
Cognitive-developmental theorists assert that cooperative efforts lead to disequilibrium and 
cognitive reorganization, which promote group goals. Finally, proponents of behavioral 
learning theory posit that cooperative efforts are influenced by extrinsic motivation to achieve 
rewards and positive reinforcement. All theories predict that cooperative learning 
environments foster higher academic achievement levels than do competitive or 
individualistic settings (Johnson et al., 1998). 

Although Slavin (1990) proposed a two-element theory of cooperative learning 
comprising positive interdependence and individual accountability, it is the five-component 
theory of D.W. Johnson, R.T. Johnson, and their colleagues (Johnson, Johnson, & Holubec, 
1991; Johnson et al., 1991a; Johnson, Johnson, & Smith, 1991b) that presently is the most 
utilized. According to this conceptualization, the following five elements are essential for 
maximizing the success of the cooperative learning endeavor: (a) positive interdependence, 
(b) face-to-face promotive interaction, (c) individual accountability, (d) social skills, and (e) 
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group processing. Positive interdependence refers to each student recognizing that she/he 
is interconnected with all other group members in such a way as the student cannot be 
successful unless all the remaining group members are. Face-to-face promotive interaction 
involves students enhancing each other’s goals by utilizing such techniques as supporting, 
praising, encouraging, and scaffolding. Individual accountability places the onus on the 
student to master the assigned task. In so doing, coat-tailing and social loafing (i.e., 
disproportionately benefiting from another’s work) is assumed to be minimized (Johnson et 
al., 1991; Johnson et al., 1991a; Johnson et al., 1991b). Social skills requires a positive 
interaction among all group members. Skills such as effective communication, building and 
maintaining trust, and constructively resolving conflicts, are emphasized. Finally, group 
processing refers to students being able to assess how well their group is working toward 
achieving its goals. As noted by Johnson and Johnson (1991), these five elements help to 
promote a successful cooperative learning experience for students. 

According to Smith, Johnson, and Johnson (1992), the variety of cooperative learning 
activities can be classified into the following three group types: informal learning groups, 
formal cooperative learning groups, and cooperative base groups. Informal learning groups, 
which are the least structured and short-term, require students to complete a task often 
associated with a lecture. Formal cooperative learning groups, which are longer in duration, 
comprise small groups established by the instructor to create a final product, such as a term 
assignment. Finally, cooperative base groups are stable, long-term, peer support groups 
created to enhance students’ learning and to increase participation in larger lecture classes. 
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Cooperative Learning at the College Level 

Since 1924, there have been more than 168 studies conducted that have compared 
the relative efficiency of cooperative, competitive, and individualistic learning on the 
achievement of individuals in college and adult settings (Johnson et al., 1998). These 
investigations indicate that cooperative learning techniques lead to higher levels of academic 
achievement than do either competitive (effect size = 0.49) or individualistic (efect size = 
0.53) methods (Johnson et al., 1998). Additionally, Qin, Johnson, and Johnson (1995), in 
a review of 46 studies at the post-secondary level, found positive effects on problem solving 
associated with the cooperative learning model in 55 of the 63 outcomes. 

However, virtually all of the cooperative learning studies undertaken on adults have 
been at the associate or baccalaureate levels, or the like. As noted by Slavin (1991), scant 
research exists in the area of cooperative learning at the graduate level. Indeed, an 
extensive review of the literature revealed only four studies examining the effects of 
cooperative learning in graduate-level research methodology courses. This is extremely 
surprising bearing in mind that (1) virtually all graduate educational programs require 
students to enroll in at least one research methodology course as a part of their degree 
programs (Onwuegbuzie, 1997); and (2) recently, there has been an increase in the number 
of research methodology instructors who utilize cooperative learning techniques in their 
courses (Onwuegbuzie & DaRos, in press). As such, little is known about the effectiveness 
of cooperative learning techniques impacting student performance at the graduate level. 
Even among the four extant investigations in this area, the findings were inconclusive. 
Specifically, Wilson (1998) found that encouraging students to work in groups when used 



Treatment By Aptitude Interactions 8 
in combination with strategies was helpful in reducing levels of anxiety among graduate 
students enrolled in educational research courses. These strategies involved addressing 
student anxiety about course content, using humor, applying statistics to real-world 
situations, and reducing fear of evaluation. Unfortunately, the cooperative techniques were 
not isolated from the other methods. Thus, it was beyond the scope of the inquiry to 
determine the individual effect of cooperative learning on statistics anxiety. 

Second, Onwuegbuzie and DaRos (in press) utilized a mixed-methodological 
equivalent-status research design (Tashakkori & Teddlie, 1998) to investigate the effects of 
cooperative learning on levels of achievement and attitudes in research methodology 
courses. These researchers found that students enrolled in classes in which cooperative 
base groups were formed (n = 81) had statistically significantly lower performance levels at 
the midpoint of the course (effect size = 0.48), as measured by the midterm examination, 
than did students who were enrolled in sections in which all assignments were undertaken 
and graded individually (n = 112) . Interestingly, although students in the cooperative 
learning groups still had lower levels of performance than did their counterparts with respect 
to the final examination, this difference was not statistically significant. No overall difference 
in course average was found between these two groups. Furthermore, qualitative (i.e., 
phenomenological) analysis of reflexive journals indicated that 70.2% of the students tended 
to have positive overall attitudes towards their cooperative learning experiences, 19.2% of , 
the students tended to have negative overall attitudes, and 10.6% tended to be ambivalent. 

Third, Courtney, Courtney, and Nicholson (1992) found no difference in statistics 
achievement between graduate students who were taught using a cooperative learning 
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method and those who were taught using a traditional method. However, their qualitative 
data suggested that cooperative learning techniques positively influenced student 
motivation, self-efficacy, level of anxiety, and sense of social cohesiveness. Fourth, most 
recently, Onwuegbuzie (in press) found a small but statistically significant relationship 
between peer orientation and achievement, with students who were more oriented towards 
cooperative learning attaining lower levels of achievement than did those who did not have 
an orientation towards cooperative learning. 

Group Composition in Cooperative Learning 

One question that has yet to be resolved at the college level in general, and at the 
graduate level in particular, is the role of group size and group homogeneity on the efficacy 
of cooperative learning partnerships. Indeed, debate exists at all levels about how to 
compose cooperative teams. With respect to the size of cooperative learning groups, 
Johnson and Johnson (2000) cautioned that (a) as the size of the group increases, 
resources needed for the group to be successful subsequently increases; (b) the shorter the 
period of time available to complete the task, the smaller the learning group should be; (c) 
the larger the group, the easier it is for them to avoid contributing their share of work (i.e., 
social loafing) (Stephan & Mishler, 1952); (d) the larger the group, the less likely that 
members will perceive their contribution to the group as being important to the group’s 
chances of success (Kerr, 1989; Olson, 1965); (e) the larger the group, the less the 
individual accountability (Messick & Brewer, 1983; Morgan, Coates, & Rebbin, 1970); (f) the 
larger the group, the more skillful the group members must be; (g) the larger the group, the 
less the interaction and communication that exists among members (Gerard, Wilhelmy, & 
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Conolley, 1968; Indik, 1965); (h) the group size is dependent on the materials available and 
the specific nature of the task; (I) the larger the group, the more difficult it is to identify any 
difficulties students have in collaborating with group members (Fox, 1985); Q) the larger the 
group, the more likely students are to strive collectively for unanimity that overshadows 
members’ motivation to examine all perspectives in a critical manner (i.e., groupthink; Bales 
& Strodtbeck, 1951; Janis, 1972); and (k) the larger the group, the stronger the positive 
independence among the members must be. 

Johnson and Johnson (2000) also noted that groups will be less effective (a) the 
larger the discrepancy between the functional group size is; (b) the less group members 
perceive their individual efforts as being crucial for group success; (c) the less the effort is 
expended by each member; (d) the more complex the group structure is, (e) the more time 
it takes for the group to coordinate efforts; (f) the less the members identify with the group; 
and (g) the less members follow the group’s norms. 

It appears that most cooperative learning groups range from two to four (Johnson & 
Johnson, 2000) or from three to five (Magney, 1997) students. Interestingly, Bray, Kerr, and 
Atkin (1978) found that the groups in their study did not solve problems appreciably more 
quickly than did the fastest problem-solver in two-person groups. Further, Watson and 
Johnson (1972) noted that in groups of more than eight or nine members, some participants 
are likely to assume passive roles. However, with the exception of these and a few other 
studies, although the general rule of thumb is that small groups are better than large groups 
(Johnson & Johnson, 2000), scant empirical research has investigated the effects of group 
size on performance. 
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Whereas some researchers advocate that homogeneous groups be formed, others 
recommend heterogeneous groups (Dalton & Kuhn, 1998). Unfortunately, scant research 
has investigated the effect of group composition on performance (Johnson & Johnson, 
2000). As such, “little is known about precisely how group composition and tasks interact 
to affect performance” (Johnson & Johnson, 2000, p. 461). According to Johnson and 
Johnson (2000), of the prevailing studies, the degree of homogeneity-heterogeneity among 
students’ demographic attributes, personal attributes, and abilities and skills has been 
assessed with respect to (a) performance on clearly-defined production tasks, (b) 
performance on cognitive or intellectual tasks, and (c) creative generation of ideas and 
decision making pertaining to ambiguous judgmental tasks. 

Two studies found performance on production tasks to be higher in cooperative 
groups whose members were homogeneous with regard to personal attributes (Clement & 
Schiereck, 1973; Fenelon & Megaree, 1971), whereas one investigation (Terborg, Castore, 
& DeNinno, 1976) found that the degree of heterogeneity pertaining to attitudes did not have 
an effect on performance on group-based land-surveying tasks. In a meta-analytic review, 
Wood (1987) found 12 studies in which objective performance results (i.e., speed and 
accuracy) was not significantly higher for mixed-gender groups than for same-gender groups 
of either gender. Similar results were reported for more complex learning tasks (Johnson, 
Johnson, Scott, & Ramolae, 1985; Peterson, Johnson, & Johnson, 1991). On the other 
hand, Cumming (1983) concluded that mixed-ability groups were more effective than were 
groups homogeneous in ability. For decision-making tasks, Hill (1982), who reviewed 
several published studies in this area, found that heterogeneous groups performed at levels 
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less than their potential. In contrast, however, Bantel and Jackson (1989) found that the 
more heterogeneous decision-making bank management teams were with regard to job 
expertise, the more frequently the bank adopted innovative initiatives. With respect to 
cognitive performance, as noted by Johnson and Johnson (2000), “there are too few studies 
on intellectual tasks to make a conclusion" (p. 459) about the impact of group heterogeneity. 
Thus, clearly, the findings pertaining to performance on clearly-defined tasks and decision- 
making tasks are contradictory. 

A few disadvantages have been reported for homogeneous groups. Specifically, too 
many members with similar thoughts and opinions may (a) induce groupthink (Janis, 1972); 
(b) induce risk-avoidant behaviors (Bantel & Jackson, 1989); (c) enhance mediocrity, 
thereby stunting effective decision-making and creative thinking; and (d) find it difficult to 
adapt to changing circumstances. However, constructing diverse groups does not 
automatically guarantee positive results. In fact, too much diversity can lead to lower group 
performance levels due to communication and organizational difficulties (Johnson & 
Johnson, 2000). Thus, it is clear that more research is needed regarding the effective of 
group composition on achievement, especially that pertaining to intellectual tasks. 

Most recently, using qualitative techniques, Onwuegbuzie and DaRos (in press) 
found that the most heterogeneous groups tended not to function as well as did 
homogeneous groups. However, this finding has not been tested empirically in research 
methodology courses. Consequently, the purpose of this investigation was to determine 
whether (a) groups with the highest mean levels of knowledge of the research process, as 
measured via mean midterm and final examination scores, produce the best cooperative 
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learning projects, as measured by the quality of research article critiques and proposals; (b) 
degree of heterogeneity is related to the quality of these group projects; and (c) size of the 
group is related to quality of output produced. 

Unfortunately, even though much research exists on the effect of cooperative 
learning, most of these studies have contained a serious analytical flaw. Specifically, in the 
majority of inquiries in this area, the treatment was given to each small group of students but 
then the individual students were inappropriately used as the unit of analysis, without taking 
into account any of the possible confounding factors (McMillan, 1999). Rather in these 
studies, it is the groups that should have formed the unit of analysis (McMillan, 1999; 
Onwuegbuzie, 2000a). In particular, in utilizing a unit of analysis at the individual level, 
researchers have failed to realize that “the idiosyncratic nature of how each group 
progresses, based on who is in the group, is likely to be a primary determinant of the results” 
(McMillan, 1999, p. 3). Because cooperative learning groups are designed such that 
students within each team influence one another, it is likely that using unit of analyses at the 
individual level to assess group efficacy leads to the independence assumption being 
grossly violated, as well as to the creation of systematic error (McMillan, 1999). As noted 
by McMillan (1999), the effect of the non-independence of observations on the unit of 
analysis is an increase in the Type I error rate, a reduction in statistical power, and a 
decrease in both internal and external validity. Therefore, it was hoped that the present 
investigation would provide more internally and externally valid findings than has been 
typically the case in cooperative learning research. 
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Method 

Participants 

Participants were 275 graduate students from a number of disciplines (e.g., 
elementary education, middle grades, secondary education, speech language pathology, 
and psychology) who were enrolled in several sections of a introductory-level research 
methodology course at a southern university over a three-year period, the majority of 
participants was female (84.0%), ranging in age from 21 to 55 ( M = 29.9, SD = 7.9), and with 
a mean grade point average of 3.64 (SD = 0.38). The racial composition was 94.2% 
Caucasian-American and 5.5% African-American. These students ( n = 275) formed 70 
groups ranging in size from 2 to 7 (M = 3.99, SD = 1 .27). The same instructor taught all 
sections of the research methodology course, thereby minimizing any implementation threat 
to internal validity resulting from differential selection of instructors (Onwuegbuzie, 2000a). 
Setting 

All graduate students enrolled in social and behavioral science degree programs were 
required to take the introductory-level educational research course. For each semester, 
which lasted for 16 weeks, class sessions were conducted for three hours, once per week. 
The fact that all classes were held at the same time in the evening (i.e., 5 pm to 8 pm) 
minimized any implementation threat to internal validity resulting from differential time of day 
(Onwuegbuzie, 2000a). 

Research proposals. One of the main requirements of the course was the completion 
of a research proposal. The objective of this proposal was to prepare students throughly to 
be able to write proposals for theses and dissertations, and for seeking external funding. As 
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such, the research proposals provided authentic assessment (Onwuegbuzie, 2000a; 
Wiggins, 1990). These research proposals were undertaken using cooperative learning 
groups. 

Research proposals had to be unique, realistic, have educational significance, and 
extend the knowledge base. A completed group-developed research proposal, which could 
represent either quantitative or qualitative research on a topic selected by the group 
members, consisted of a title, a one-page proposal summary, an introduction section, a 
review of the related literature, a methodology section, an analysis section, a bibliography, 
and an appendix section including a biography of each group member, timetable, budget, 
consent form(s), and author-designed instrument(s). Each group proposal had to be typed, 
following guidelines specified by the American Psychological Association ([APA],1994). The 
writing style of each proposal (e.g., grammar, punctuation, clarity, and application of the APA 
(1994) criteria) also was assessed. All proposals had to include an in-depth review of the 
literature, and thus extensive library usage was required. Indeed, although many research 
methodology instructors appear to require what could be conceptualized as a mini-proposal, 
the research proposal in this course was required to be comprehensive, containing a 
minimum of 20 references--the majority of which were to be obtained from refereed research 
journals. 

Historically, over the years, research proposals in this course typically ranged from 
25 to 40 pages, with the literature review section usually ranging from 5 to 15 pages. 
Students in each group were encouraged to begin the process of developing their research 
proposals from the first class meeting. Moreover, groups were required to formulate their 
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research questions by the second class meeting and to start obtaining literature sources by 
the third class meeting. 

Article critique. The second major course requirement that was undertaken via 
cooperative learning groups involved a detailed written critical evaluation of a published 
research report (i.e., article critique). The primary goal of the article critique was to provide 
an opportunity for students to develop skills in evaluating published research articles utilizing 
principles of the scientific method. In order to facilitate this process, students were required 
to select several articles to critique, and to bring them to the second class meeting for advice 
from the instructor as to their appropriateness (i.e., article content utilizing principles of the 
scientific method). Furthermore, students were required to make their final selection as to 
which article to critique by the third week of the semester The article critiques provided 
performance assessment (Hutchinson, 1995; Onwuegbuzie, 2000b). 

Formation of cooperative learning groups. On the first day of class, students, in turn, 
were asked to introduce themselves to the class, delineating their major, educational 
attainments and aspirations, current professional status, and interests. Following these 
introductions, students were asked to form groups comprising 3-6 students. Group 
formation was guided by asking students to choose group members based on similar 
majors, professional background, and proximity to each other’s homes. These criteria for 
group assignment were not directly related to aptitude or ability. Such assignment of groups 
by preferences is referred to as a modified stratified random assignment (Johnson & 
Johnson, 2000). 

Nine groups involved pairs. These pairs were formed when two students lived close 
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to one another, but a significant distance from other students in the class, or when the major 
of two students in the class (e.g., music) was different to all other class members. Two 
groups of seven students also were formed because these groups represented students 
who were admitted to their degree program as a cohort. Members of these seven-group 
teams lived close to one another but a significant distance (more than 90-minute drive) from 
the university they attended (i.e., where the current investigation took place). 

Base groups. The cooperative learning group that was utilized involved the use of 
base groups (Smith et al., 1992). The aim of these base groups was to promote stable 
membership whose foremost responsibility was to provide each member of the group the 
support, encouragement, and assistance as needed to comprehend course content. In 
addition, the cohesiveness provided by membership in the group was (a) to promote the 
successful completion of the course assignments and (b) to prepare students for the in- 
class individual examinations. Students were encouraged to stay together during the entire 
course. Additionally, students were expected to takes notes for and to distribute any 
instructor handouts to any group member who was unable to attend a class session. That 
is, students were expected to provide peer tutoring to their absent group members. Although 
they were allowed to change groups if any conflicts or unresolvable problems arose among 
group members, no student requested such a change. Students were asked to exchange 
telephone numbers and e-mail addresses and information about their schedules so that they 
could meet outside class. Students were encouraged to take notes for and to collect class 
handouts for any absent group members, as well as to provide peer tutoring of any new 
concepts that were covered by the instructor during the missed session. Each base group 
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undertook one research proposal and one article critique. 

The instructor informed students of the following basic group skills: group activities 
should be distributed as equally as possible, or at least according to the strengths of group 
members; students should respect the opinions of all group members, no students should 
dominate group discussions; and every student should be aware of all tasks undertaken by 
group members and be prepared to provide constructive criticism. Students were not 
assigned specific group roles; however, they were presented with different models for the 
division of labor (e.g., each student writing a section of the research proposal and article 
critique; each student individually undertaking all sections of these assignments and then 
comparing their work with other group members with the goal of merging all responses). 
Additionally, as discussed by Garfield (1993), students were made aware of the importance 
of assigning different roles at each group meeting (i.e., moderator/organizer, summarizer, 
recorder, strategy suggester, seeker of alternative methods, mistake manager, and an 
encourager) that prevented any one person from undertaking a disproportionate amount of 
the work. This discussion of group roles was undertaken in an attempt to maximize positive 
interdependence, promotive interaction, and social skills, as recommended by Johnson and 
Johnson (1991). 

Course organization. The first part of each class period typically consisted of a review 
of the material presented the previous session, and the middle portion of each class lesson 
generally involved the presentation of new material. All students were provided with a 
complete set of the instructor’s lecture notes at the beginning of the course. However, 
instead of a lecture-based review of the material, each base group reviewed the material 
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that was presented earlier by the instructor. During this phase, students rearranged desk- 
chairs into groups within the classroom. While students worked in groups, the instructor 
observed, answered questions posed by students, facilitated discussion among all group 
members, identified and praised group successes, and informed the class of any insights 
gained from circulating among the groups. These techniques were utilized by the instructor 
in an attempt to enhance positive interdependence, promotive interaction, and social skills 
(Johnson & Johnson, 1991). As time permitted, students in the cooperative groups were 
given class time toward the end of the period to discuss their research proposals and article 
critiques and to engage in group processing (Johnson & Johnson, 1991). Again, the 
instructor served as a facilitator. 

Due to the comprehensiveness of the article critique and the research proposal, the 
instructor attempted to make himself as available as possible to all students outside class 
time and office hours, encouraging them to contact him at his home between 10 am and 10 
pm, on any day of the week (including weekends and holidays), if they had any questions 
about the assignments. Many students took advantage of this opportunity. In fact, it was 
common for the instructor to receive a conference call involving some or all members of a 
cooperative learning group. Indeed, on many occasions, the cooperative group that had 
telephoned their professor used a speaker phone in order that all group members could hear 
their instructor’s responses to their questions. 

Instruments 

A scoring rubric was used to evaluate the research proposals and article critiques (cf. 
Wilson & Onwuegbuzie, 1999), with detailed feedback provided by the professor. Students 
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received group scores for these assignments. Conceptual knowledge, which involved 
students' knowledge of research concepts, methodologies, and applications, was measured 
individually via comprehensive written midterm and final examinations. These examination 
forms consisted of open-ended questions, involving items which required knowledge of the 
research process (e.g., “What is the difference between inductive and deductive research?"). 
All of the items in the midterm examination form pertained to content from the first half of the 
course and were chosen from the instructor's item bank to ensure that the examination was 
typical of past examinations given by the instructor. The final examination also was 
constructed by the course instructor and paralleled the format of the midterm examination, 
yet covered the complete course content. Both the midterm and the final examination were 
administered under untimed conditions, and were scored on a 100-point scale by the 
instructor, using a key that specified the number of points awarded for both correct and 
partial-credit answers. 

Additionally, students were required to complete a peer evaluation form to assess the 
level of cooperativeness of their group members. This peer evaluation form consisted of a 
10-point rating scale containing 10 items, with scores ranging from 10 to 100. High scores 
on the scale indicated a favorable cooperative rating from a fellow group member. Items on 

the peer evaluation form included the following: (a) “ was willing to do his/her fair share 

of the work”; (b) “ listened to the opinions of others in the group”; and (c) “ 

provided assistance to other members of our group.” For each student, scores on the peer 
evaluation form were averaged across team members to obtain an overall cooperativeness 
rating. Thus, the score for the article critique and research proposal for the group to which 
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the student belonged was weighted by her/his participation score, such that if a student 
received 100% of the participation points available, her/his individual score would be exactly 
equivalent to the group score. If the student receives 90% of the participation points 
available, her/his individual score will be worth 90% of the group score, and so forth. The 
midterm and final examinations, as well as the peer evaluation forms, were administered in 
an attempt to ensure individual accountability (Johnson & Johnson, 1991 ). 

Results 

The individual midterm scores ranged from 51 to 100 (M = 79.54, SD = 11.21), 
whereas the individual final examination scores ranged from 55 to 100 (M = 82.79, SD = 
9.33). Further, the mean midterm scores for each group ranged from 62.67 to 99.00 ( M = 
79.35, SD = 7.08). Additionally, the group variances of the midterm scores ranged from .71 
to 18.34 ( M = 8.73, SD = 4.28). The mean final scores for each group ranged from 69.33 
to 93.33 ( M = 82.59, SD = 5.10), whereas the group variances of the final scores ranged 
from 2.63 to 19.09 (M = 8.63, SD = 3.69). Interestingly, the relationship between the mean 
group midterm and the mean group final examination scores was statistically significant (r 
= .63, p < .05) and large (Cohen, 1988). 

Using group as the unit of analysis revealed statistically significant moderate positive 
relationships between the mean midterm scores and scores on the article critiques (r= .26, 
p < .05) and proposals (r= .35, p < .01). Similarly, moderate-to-large (statistically significant) 
positive relationships were found between the mean final scores and scores on the article 
critiques (r = .31 , p < .01 ) and proposals (r = .46, p < .001 ). 

Additionally, a statistically significant positive relationship was found between degree 
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of group heterogeneity at the midterm level (i.e., group aptitude) and scores on the group 
article critique (r = .25, p < .05), although no relationship was found between degree of 
midterm heterogeneity and scores on the group proposal (r = .11, p > .05). Also, no 
association was noted between degree of group heterogeneity at the final level (i.e., group 
ability) and scores on both the group article critique (r = .05, p > .05) and proposal (r = .18, 
p > .05). 

No relationship was found between group size and group performance on the article 
critique (r= .14, p > .05) and proposal (r= .05, p > .05). However, a trend analysis (removing 
the two seven-group teams from the analysis) revealed no linear (F [1 , 67] = 1 .99, p > .05), 
cubic (F [1 , 67] = 0.31 , p > .05), or quartic (F [1 , 67] = 1 .90, p > .05) trend. Conversely, a 
quadratic trend (F [1 , 67] = 4.43, p < .05) emerged, with the article critique scores peaking 
at a group size of 6 (i.e., M = 95.50, SD = 5.21). However, Scheffe’s test revealed no 
pairwise differences in article critique group means, although it should be noted that this lack 
of statistically significant difference between group means may have been the result of the 
low statistical power, bearing in mind that there was a 12-point difference between the three- 
group and the six-group teams, in favor of the latter. Figure 1 illustrates the quadratic trend. 
Interestingly, no linear (F [1 , 67] = 1 .1 1 , p > .05), quadratic (F [1 , 67] = 0.01 , p < .05), cubic 
(F [1 , 67] = 3.98, p > .05), or quartic (F [1 , 67] = 1 .37, p > .05) trend was found with respect 
to the scores on the research proposals. 



Insert Figure 1 about here 
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Most interestingly, a treatment (i.e., group heterogeneity level at the midterm level) 
x aptitude (i.e., mean midterm group performance) interaction (F [1, 66] = 4.73, p < .05, of 
= .27) was found with respect to group article critique scores. Moreover, this interaction was 
ordinal in nature, in which the difference in article critique scores between cooperative 
learning groups in the upper half of the distribution with respect to midterm scores (i.e., high- 
aptitude groups) ( M = 88.72, SD = 9.87) and those in the lower half of the distribution (i.e., 
low-aptitude groups) ( M = 78.18, SD = 8.47) was statistically significantly larger for groups 
in lower half of the midterm score distribution with respect to heterogeneity (i.e., low- 
heterogeneous groups) than the difference between high-aptitude groups ( M = 90.00, SD 
= 5.49) and low-aptitude groups ( M = 87.72, SD = 6.82) for high-heterogeneous groups. 

In addition to the aptitude x treatment interaction observed for article critique scores, 
both the aptitude main effect (F[1, 66] = 11.37, p < .001, a/ 2 = .42) and the heterogeneity 
main effect (F [1 , 66] = 8.10, p < .01 , oj 2 = .35) were statistically significant. Both these effect 
sizes were large (Cohen, 1 988). With respect to the former main effect, high-aptitude groups 
(M = 89.32, SD = 8.02) produced significantly better article critiques than did the low- 
aptitude groups ( M = 83.09, SD = 8.97). With respect to the latter, the quality of article 
critiques was higher for high-heterogeneous groups ( M = 88.79, SD = 6.24) than for low- 
heterogeneous groups ( M = 83.60, SD = 10.54). 

With respect to research proposals, no treatment (i.e., group heterogeneity level at 
the midterm level) x aptitude (i.e., mean midterm group performance) interaction (F [1 , 66] 
= 0.66, p > .05, a f - .02) was found. Similarly, no heterogeneity main effect (F [1 , 66] = 0.80, 
p > .05, a? = .02) emerged. However, there was an aptitude main effect (F [1, 66] = 4.65, 
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p < .05, a? = .1 1). This effect was small (Cohen, 1988). 

Discussion 

A decision that faces every college instructor who implements cooperative learning 
in his/her class relates to group composition. Specifically, these instructors must decide 
upon the optimal group size and composition within the context of the assigned material or 
project. Unfortunately, the research base does not make clear the optimal group size or 
group composition at any level of the educational process (i.e., primary, secondary, or 
tertiary). The few studies that have examined the role of group size and composition in 
cooperative learning have yielded mixed findings (Dalton & Kuhn, 1998). Thus, the purpose 
of the present inquiry was to investigate the effect of these factors on the academic 
performance of cooperative groups enrolled in a graduate-level research methodology 
course. Surprisingly, although researchers have analyzed the effectiveness of cooperative 
learning in college classrooms since 1924, there has been a paucity of such studies at the 
graduate level. 

Several important findings emerged from the present research. First and foremost, 
contrary to Johnson et al. (1991a), who assert that allowing students to select their own 
groups leads to homogeneous groups, the student-selected groups in the current 
investigation were extremely heterogeneous in their performance levels, with the mean 
group standard deviation for both midterm and final scores being close to nine percentage 
points. In other words, each cooperative learning group, on average, contained students 
who differed in individual examination performance scores by nearly one grade. Indeed, 
students in some groups varied in individual performance by as much as two grades, with 
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Grades of “A,” “B,” and “C” being attained, for example, by students in these groups. The 
fact that many of the groups were heterogeneous may have resulted from the selection 
criteria, which were based on academic major, professional background, and proximity to 
each other’s homes, and not based on aptitude and ability considerations. Thus, it appears 
that use of these three criteria (i.e., a modified stratified random assignment procedure) was 
effective in producing heterogeneous groups. Nevertheless, the fact that the group mean 
midterm examination scores ranged from a Grade “D” (i.e., 62.8%) to Grade “A” (i.e., 99.0%) 
suggests that the cooperative groups differed substantially with respect to performance 
levels of the individual group members. 

One of the most important results of the present inquiry was the statistically significant 
moderate-to-large positive relationships between the mean midterm scores and scores on 
the article critiques and proposals, as well as those between the mean final scores and 
scores on the article critiques and proposals. These combined results suggest a “Matthew 
effect,” whereby groups that contained higher-achieving students on an individual level 
tended to produce better group outcomes than did their lower individual-achieving 
counterparts. Interestingly, the large positive relationship between midterm examination 
scores and final examination scores provides further support for the Matthew effect with 
respect to group outcomes. 

The Matthew effect was coined by Merton (1 968) after the Biblical statement that “For 
unto every one that hath shall be given, and he shall have abundance: but from him that 
hath not shall be taken away even that which he hath” (Gospel according to Matthew, 
XXV:29). Merton described the Matthew effect with respect to scientific productivity as 
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representing the accumulation of greater increments in recognition for specific scientific 
works to scientists of notoriety, and the withholding of such recognition from scientists who 
have not yet established themselves in the field. Merton’s idea of the “rich getting richer’’ was 
subsequently observed and described in educational settings by Walberg and his associates 
(Walberg, Strykowski, Rovai, & Hung, 1984; Walberg & Tsai, 1983). In particular, Walberg 
and Tsai (1983) described a “fan-spread” when educational outcomes are plotted against 
time such that “rates of gain are relative and proportional to initial endowment” (p. 361). 
Later, Stanovich (1986) suggested that the Matthew effect should be considered in 
understanding the role of initial reading level on later reading performance. 

According to Walberg et al. (1984), the Matthew effect implies that “rather than the 
one-way causal directionality usually assumed in educational research, reverberating or 
reciprocal states may cause self-fulfilling or self-reinforcing causal processes that are highly 
influential in determining educational and personal productivity" (p. 92). In the current 
research, the fact that groups consisting of individually higher-achieving students tended to 
produce better article critiques and research proposals than did other cooperative learning 
teams may be the result of a self-fulfilling prophecy in which higher individual-achieving 
groups possess higher levels of academic motivation and self-esteem, and other affective 
components, than do their counterparts. Indeed, Onwuegbuzie and DaRos (in press) 
reported that most students in cooperative learning groups experience increases in 
motivation, persistence, self-esteem, self-efficacy, anxiety, social cohesion, problem solving 
adeptness, and metacognitive awareness. However, it is likely that these gains were largest 
for the higher individual-achieving groups. Also, it is possible that students in the highest 
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individual-achieving groups were the least likely to procrastinate on their group assignments. 

With respect to Johnson et al.’s (1991a) assertions regarding the five elements that 
underlie successful cooperative learning groups, it is clear that, at the very least, the highest- 
scoring groups displayed the highest individual accountability. That is, members of these 
groups were the most responsible in completing the assignments. It is also possible that 
these successful groups tended to enforce the other four elements (i.e., positive 
interdependence, face-to-face promotive interaction, social skills, and group processing) of 
cooperative learning to a greater extent than did the less successful teams. However, it was 
beyond the scope of the present investigation to determine this. Consequently, future 
research, utilizing qualitative research methodologies should explore the role of these four 
elements in promoting a Matthew effect in cooperative learning groups. 

The moderate positive relationship found between degree of group heterogeneity at 
the midterm level (i.e., group aptitude) and scores on the group article critique, as well as 
the large Heterogeneity effect (vis-a-vis individual midterm scores) observed for article 
critique scores in favor of high-heterogeneous groups, contradicts the qualitative 
observations of Onwuegbuzie and DaRos (in press); however, these current results are 
consistent with Johnson et al. (1991a), who recommend that instructors maximize the 
heterogeneity of groups by placing low-, medium-, and high-achieving students in the same 
cooperative group. According to these authors, “more elaborative thinking, more frequent 
giving and receiving of explanations, and greater perspective taking in discussing material 
seem to occur in heterogeneous groups, all of which increase the depth of understanding, 
the quality of reasoning, and the accuracy of long-term retention” (pp. 60-61). The findings 
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from the present investigation clearly indicate that research methodology instructors should 
incorporate heterogeneous cooperative learning groups in their instructional format at the 
graduate level. 

The treatment x aptitude interaction found with respect to the article critiques 
indicated that the difference in quality of article critiques produced between the high-aptitude 
groups and low-aptitude groups was greater for the low-heterogeneous groups than for the 
high-heterogeneous groups. Thus, the Matthew effect was strongest for the more 
homogeneous groups, thereby providing incremental validity to the contention that 
instructors should maximize the heterogeneity of cooperative learning groups (Johnson et 
al., 1991a). Interestingly, the treatment x aptitude interaction that emerged in the present 
investigation is consistent with Onwuegbuzie and DaRos (in press), who, using qualitative 
techniques, found evidence of such an interaction. 

That a Heterogeneity effect and a Treatment by Aptitude effect was found with 
respect to the quality of group article critiques, but that these effects were not noted with 
regard to the quality of group research proposals, may have arisen because the article 
critique was a more complex assignment than was the research proposal. As noted by 
Burns and Grove (1987), critiquing a research article involves first identifying the study 
elements and understanding the nature, significance, and meaning of both implicit and 
explicit components. Second, article critiques require the interpretation of meanings of the 
terms and concepts used in the report in the same way as the researcher(s) used them. 
Third, it is important that students have extensive knowledge of what each step of the 
research process comprises in order to evaluate the extent to which the article follows this 
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process. Fourth, it is imperative that the student is able to identify the expressed and 
unexpressed assumptions of the researcher, as well as to examine the abstract dimensions 
of the study. In doing this, students must be cognizant of the links between the elements of 
the study, as well as links between elements of the study and previous research. Finally, 
conceptual clustering (Werley & Fitzpatrick, 1985) must be undertaken, which maximizes 
the meaning attached to research findings, highlights gaps in the knowledge base, and 
generates new research questions. The above five steps of the critique process, namely, 
comprehension, comparison, analysis, evaluation, and conceptual clustering, must occur 
in sequence, with each step presuming accomplishment of the previous step (Burns & 
Grove, 1987). Thus, critiquing an article involves even more complex processing than 
acquiring conceptual understanding of research or being able to write a research proposal. 
Indeed, evaluation is the highest level of learning in Bloom's taxonomy of cognitive 
objectives (Bloom, 1956). This may help to explain why the treatment x aptitude effect 
occurred only for article critiques. 

With respect to group size, no trend was found with respect to the scores on the 
research proposals; however, a quadratic trend was observed for scores on the article 
critiques. Although replications are needed to assess the generalizability of these findings, 
these results suggest that the size of the group may make a difference, especially when the 
complexity of the group task (i.e., article critique) is at a premium. Interestingly, groups 
containing six students, on average, obtained the highest scores on their article critiques. 
Yet, it should be noted that this was the largest group compared. Thus, it is not clear 
whether groups of this size produced the best quality article critiques because they 
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displayed the greatest group cohesion or because they had the highest probability of having 
one or more functional members who did the bulk of the work. In other words, it is beyond 
the scope of the present investigation to determine whether the degree of social loafing was 
more prevalent in six-student groups than in other combinations of groups. Indeed, Watson 
and Johnson (1972) asserted that in very large groups, a few members are apt to dominate 
and the remaining members are likely to assume passive roles. Moreover, as the size of 
cooperative groups increases, (a) members are less likely to deem their own personal 
contributions to the group as being essential to the group’s level of achievement (Kerr, 
1989; Olson, 1965); (b) individual accountability is less likely to prevail (Messick & Brewer, 
1983); (c) individual group members typically communicate less frequently, thereby affecting 
the amount and quality of information utilized to make group decisions (Gerard et al., 1968; 
Indik, 1965); and (d) groupthink is more likely to prevail (Gerard et al., 1968; Rosenberg, 
1961). However, the fact that the relationship between group size and scores on the article 
critique in the present investigation was non-linear (cf. Figure 1), with two-student groups 
obtaining the second highest mean scores, suggests that the ratio of the functional group 
size to the actual group size may not have increased as a function of actual group size. In 
any case, future inquiries should ascertain the relationship among group size, functional 
size, level of positive interdependence. Qualitative research techniques would be particularly 
useful in such studies. 

In summary, the Matthew effect, the Heterogeneity effect, and the treatment x 
aptitude interaction appear to prevail when cooperative learning groups are utilized in 
research methodology classes. This suggests that instructors of these courses should be 
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cognizant of the potential debilitative and facilitative roles that group composition plays in 
cooperative or purposively-formed groups. Thus, it appears that in assigning students to 
groups in graduate-level research methodology classes, instructors should consider forming 
the cooperative learning groups either randomly or purposively (e.g., modified stratified 
random assignment procedure), using criteria (e.g., major, profession, and proximity to each 
other’s homes) that are not directly related to aptitude or ability. Even more importantly, the 
results of the present investigation indicate that merely assigning students to groups is 
insufficient; instructors must constantly monitor that the five elements of cooperative learning 
are being adhered to by each group, as well as monitor the impact of group composition in 
an attempt to minimize the Matthew effect, the Heterogeneity effect, and the Treatment x 
Aptitude interaction that were observed in the present investigation. 

Although there is a myriad of studies documenting the positive effect of cooperative 
learning with respect to several educational outcomes relative to more competitive and 
individualistic classroom environments, little is known about the conditions under which 
cooperative learning is maximized. The present study suggests that relatively large (n = 6) 
and heterogeneous groups represent optimal conditions. However, because this 
investigation was conducted in a geographically-restricted area, it is not clear the extent to 
which the Matthew effect, the Heterogeneity effect, and the treatment x aptitude interaction 
generalize to other settings. Thus, replications are needed. Indeed, these replications should 
be undertaken using other types of college classes (i.e., non research-based classes) and 
utilizing different levels of college students (e.g., doctoral, specialist, Master’s, and 
undergraduates). Additionally, more such investigations are needed at other types of 
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colleges (e.g., community colleges), as well as at primary and secondary school levels. 
Further, future research should determine whether the Matthew effect prevails when 
students are purely randomized to cooperative learning groups. An important feature of all 
empirical-based replication studies in this area is that the groups themselves rather than the 
individuals are utilized as the major unit of analysis, as was undertaken in the present 
investigation. Using groups as the unit of analysis would minimize the possibility of the 
statistical independence being violated and systematic error being created (McMillan, 
1999). It is only by establishing a large database that validly documents the effect of group 
composition on cooperative learning can we come close to understanding the educational 
and psychological processes that underlie this promising method of instruction. 
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Figure Caption 

Figure 1. Mean Scores for the Article Critiques as a Function of Group Size. 
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